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(54) IMAGE ENCODING DEVICE CONSIDERING SLAVE PICTURE DISPLAY 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide an image 
encoding device which does not bring about so much 
image deterioration due to change of the position and 
size of slave picture display while reducing code quantity 
per picture without producing a part where an image 
does not exist in a decoded image even though the 
position and size of a slave picture is changed and 
erased after starting image data transmission. 
SOLUTION: Slave picture display information of an 
image decoding device Tb1 is received from the device 
Tb1 connected through a line OCT, a controlling part 4a1 
controlling the code quantity of each divided area of an 
input image 3a is also provided, and the part 4a 1 
performs code quantity reduction control that reduces 
the code quantity of the area of the input image 3a 
corresponding to the slave picture display part 8a1 of 
the device Tb1 received on the basis of slave picture 
display information. 
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[Claim(s)] 

[Claim 1]A child screen-display-information receive section which receives child screen display 
information of an image decoding device to an image decoding device which is an image encoding 
apparatus which codes an inputted image, and was connected via a circuit, An image encoding 
apparatus, wherein it has a control section which controls a code amount for every field by 
which an inputted image was divided and said control section performs code amount reduction 
control which reduces a code amount of a field of an inputted image equivalent to a child display 
part of an image decoding device receivad in said child screen-display-information receive 
section. 

[Claim 2]The image encoding apparatus according to claim 1 , wherein code amount reduction 
control by said control section quantizes data of said child screen display region by bigger 
quantization step width than other portions. 

[Claim 3]The image encoding apparatus according to claim 1 , wherein code amount reduction 
control by said control section codes data of said child screen display region by frequency lower 
than other portions. 

[Claim 4]The image encoding apparatus according to claim 1 coding it after code amount 
reduction control by said control section filters data of said child screen display region. 
[Claim 5]An image encoding apparatus given in any 1 paragraph of claims 1 thru/or 4, wherein 
code amount reduction control by said control section enables setting out or change of a code 
amount of said child screen display region data arbitrarily. 
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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]This invention relates to the image encoding apparatus which codes an 
inputted image and transmits, and the image decoding device which superimposes a child screen 
on the screen which decoded coded image information. 
[0002] 

[Description of the Prior Art]There are some which are indicated to JP,4-159890,A in the image 
encoding apparatus and image decoding device in consideration of a conventional child screen 
display. When this shortens the transmission time at the time of transmission and reception of 
image data, the portion of the child screen position of a mating terminal is made paying attention 
to image display not being carried out. Namely, in the numerals image encoding apparatus which 
codes image data and transmits. It transmits, after removing beforehand about the image data of 
the child screen position portion of the mating terminal to apply, and while the image decoding 
device of a mating terminal decodes the picture information which carried out partial removal in 
the child screen position, a desired child screen is superimposed and displayed on the portion of 
the child screen position. Hereafter, it explains in detail, referring to drawing 6 and 7. 
[0003] Drawing 6 (a) is the block diagram in which pictorial-communication-equipment (terminal) 
Ta carried out the outline. 

It has the image encoding apparatus ICa, the image decoding device IDa, the multiplexing 
decollator 5a, and the operation input means 17a. 

[0004]The video input section 2a into which said image encoding apparatus ICa inputs the 
picture la, It has the coding processing section 6a which codes the image data 7a which 
removed the child screen portion 8a by the control section 4a later mentioned from the inputted 
image data 3a, and the control section 4a which controls processing by said coding processing 
section 6a. 

[0005]Said image decoding device IDa is provided with the video output part 12a which inputs 
and superimposes the image 14a for child screen display, and outputs the multi-image 15a from 
the decoding processing section 9a which inputs and decodes received data from the 
multiplexing decollator 5a, and its decoded image 10a and said video input section 2a. 
[0006] Drawing 6 (b) is the block diagram in which pictorial-communication-equipment (terminal) 
Tb carried out the outline, and is the same composition as the above mentioned terminal Ta, and 
two-way communication is possible for it via terminal Ta and circuit CCT. While giving identical 
codes at terminal Ta and an identical configuration and omitting explanation, in order [ of 
explanation ] to perform distinction with said terminal Ta for convenience, the numerals a of an 
end are changed into the numerals b, and are shown. 

[0007]Next, the case where picture [A] 1a is transmitted from the image encoding apparatus ICa 
of terminal Ta, and the decoding device IDb of terminal Tb receives, and superimpose picture [B] 
1b, and output displaying of the outputted image 15b is carried out is explained. 
[0008] First, child screen information, such as a position of the child screen which ****** 
specified using the **** input means 17b of terminal Tb, and a size, is transmitted to the control 
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section 4a of the image encoding apparatus ICa before transmission of image data via the 
multiplexing decollator 5b, circuit OCT, and the multiplexing decollator 5a of the image decoding 
device IDb. 

[0009]The image encoding apparatus ICa which received child screen information, After removing 
the portion 8a corresponding to the child screen position of the image decoding device IDb using 
the control section 4a among the pictures 3a inputted from the video input section 2a which is 
an input means, the image data 7a is transmitted to the image decoding device IDb via the 
multiplexing decollator 5a, circuit CCT, and the multiplexing decollator 5b. 
[001 0]The image decoding device IDb which received image data decodes the image data using 
the decoding processing section 9b, and uses it as the decoded image 10b. And the image 14b 
which reduced the image 3b for [ which was inputted using video input section 2b ] 
superimposing is superimposed by the video output part 12b to this decoded image 10b in said 
child screen position, and the multi screen 1 5b is outputted. 

[001 1]When outputting the muiti screen 1 5a from the image decoding device IDa of terminal Ta, 
the flow of the image data described above using the image encoding apparatus ICb of terminal 
Tb and the operation input section 17a of terminal Ta and child screen information will become 

[0012]The coding mode using inter-frame motion prediction and orthogonal transformation as 
generally show drawing 7 a TV phone and the image coding system in the coding processing 
sections 6a and 6b used by a video conference system as a low bit rate coding method is used. 
Hereafter, the coding processing section 6a is explained and explanation is omitted about the 
coding processing section 6b of an identical configuration. 

[0013]The coding processing section 6a shown with the block diagram of drawing 7 is provided 
with motion estimate and the compensation part 609, the orthogonal transformation section 603, 
the quantizing part 604, the inverse quantization part 605, the inverse-orthogonal-transformation 
part 606, the subtraction part 602, the adder unit 607, the frame memory 608, and the variable 
length coding section 610. 

[001 4] First, use front image data for the inputted image data 3a ( drawing 6 ) inputted from said 
video input section 2a, and the time target stored in the frame memory 608, and inter- frame 
motion prediction is given to him by motion estimate and the compensation part 609, The 
difference of the inputted image data 3a ( drawing 6) and a predicted value is searched for by the 
subtraction part 602. 

[0015]The difference of the predicted value calculated by said subtraction part 602 is changed 
into the coefficient in which orthogonal transformation was carried out by the orthogonal 
transformation section 603. Orthogonal transformation is processed by 8x8 pixel units called a 
block. 

[001 6]After being quantized by the quantizing part 604, variable length coding of said coefficient 
by which orthogonal transformation was carried out is carried out by the variable length coding 
section 610. 

[0017]Variable length coding also of the motion vector called for by motion estimate and the 
compensation part 609 is carried out by the variable length coding section 610. 
[0018]If it generally quantizes by big quantization step width in the case of quantization, the 
amount of information is compressible, but a quantization error becomes large and image quality 
deteriorates. On the contrary, if it quantizes by small quantization step width, the amount of 
information is seldom uncotnpressible, but a quantization error is small and image quality 
improves. The information by which variable length coding was carried out will be transmitted to 
a decoding processing section via a multiplexing separation part and circuit CCT. 
[0019] 

[Problem(s) to be Solved by the Invention]However, in the image encoding apparatus and image 
decoding device in consideration of a conventional child screen display. If elimination of change 
which makes the position and size of a child screen small, or a child screen is performed, In 
order that there is a problem that the portion which a picture does not have will be made, and an 
image encoding apparatus may code difference with the image data transmitted immediately 
before and may generally transmit until it receives the following image data and is displayed, In 
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order to code a portion without a picture, and to use good image quality in short time, the code 
amount of the data per picture will become large. If it avoids that the code amount of the data 
per picture becomes large, Having to carry out multiple image reception until it became good 
image quality, in order to code with a small code amount per picture, the portion without a 
picture had produced the problem that the state where image quality is bad will continue for a 
long time as compared with other portions. 

[0020]This invention was made in order to cancel the aforementioned problem, and it is *#**. 
The purpose is providing an image encoding apparatus with little image quality deterioration by 
the position of a child screen display, change of a size, etc., decreasing the code amount per 
screen without the portion which does not have an image into a decoded image arising, even if it 
performs the position of a child screen, change of a size, or elimination of a child screen after a 

[0021] 

[Means for Solving the Problem]This invention has the next composition in order to attain the 
above-mentioned purpose. A child screen-display-information receive section which receives 
child screen display information of an image decoding device to an image decoding device which 
the 1 st gist of this invention is an image encoding apparatus which codes an inputted image, and 
was connected via a circuit, It has a control section which controls a code amount for every field 
by which an inputted image was divided, and there is a control section in performing code 
amount reduction control which reduces a code amount of a field of an inputted image equivalent 
to a child display part of an image decoding device received in a child screen-display-information 
receive section. 

[0022]Quantizing data of a child screen display region by bigger quantization step width than 
other portions has code amount reduction control by a control section which described the 2nd 
gist of this invention above. 

[0023]Coding data of a child screen display region by frequency lower than other portions has 
code amount reduction control by a control section which described the 3rd gist of this invention 

[0024]Coding, after filtering data of a child screen display region has code amount reduction 
control by a control section which described the 4th gist of this invention above. 
[0025]It is an image encoding apparatus given in any 1 paragraph of claims 1 thru/or 4, wherein 
code amount reduction control according [ the 5th gist of this invention ] to said control section 
enables setting out or change of a code amount of said child screen display region data 
arbitrarily further again. 

[0026]According to this invention, information on a child screen display position and a size of an 
image decoding device connected via a circuit is received, By carrying out code amount 
reduction control of the code amount of a field equivalent to a child screen of an image decoding 
device of a picture inputted by an input means so that it may become less than other fields, a 
code amount per screen will be decreased and a transmission time will be shortened, being [ no 
image ]-in position which had child screen before in decoded image **** also when a child 
screen display position and a size of an image decoding device change — things do not happen. 
When a display position and large **** information on a new child screen are received while an 
image encoding apparatus had transmitted image data, Image quality of a portion which realized 
reduction continuously by controlling a code amount of a field of an inputted image equivalent to 
a new child display part to become less than other fields, and had a child screen before in a code 
amount per screen improves. Code amount reduction control by a control section is enabling 
setting out or change of a code amount of said child screen display region data, or a decrement 
of a code amount arbitrarily, Since a setting variation of an operator becomes possible at 
arbitrary stages (beforehand inside of **********), adjustment of image quality, etc. are suitably 
united with taste, for example, and it becomes possible, and becomes a more user-friendly image 
encoding apparatus. 
[0027] 

[Embodiment of the Invention]Hereafter, with reference to 3, a 1st embodiment of this invention 
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is described in detail from drawing 1 . As described above, although an image encoding apparatus 
and an image decoding device are explained using what is united in many cases and is united also 
in this embodiment, they are not limited to this. Identical codes are given to the above mentioned 
pictorial communication equipment (terminal) Ta and Tb and identical configuration, and 
explanation is omitted. 

[0028]Image encoding apparatus ICa, of pictorial-communication-equipment (terminal) Ta, 
shown in drawing 1 (a), Input the picture 1a transmitted from the video input section 2, and it is 
considered as the image data 3a, The image data 3a is coded by coding processing section 6a,, 
and the image data and other data are transmitted to image decoding device IDb, of pictorial- 
communication-equipment Tb, which is multiplexed by the multiplexing separation part 5a, and is 
shown in drawing 1 (b) via circuit OCT. Said circuit CCT uses the data transmission media in 
which the data communications of a cable or radio are possible. 

[0029]Coding processing section 6a, which constitutes image encoding apparatus ICa,, The 
portion corresponding to the child screen position of an image decoding device is not removed 
like before, When coding the image data 3a, and child screen information, such as a display 
position, a size, etc. of a child screen, is received from image decoding device IDb,, by control- 
section 4a,. Coding processing section 6a, is controlled to decrease the code amount of the 
child screen area 8a, as compared with field 7a, of the other same area. 
[0030]That is, when coding processing section 6a, newly receives child screen information from 
image decoding device IDb,, the child screen information currently held by the storage memory 
which control-section 4a, does not illustrate is updated, and the newest child screen information 
is used for control of coding processing section 6a,. And the image data which lessened the 
code amount of the child screen area 8a, as compared with field 7a, of the other same area, and 
other data are multiplexed by the multiplexing separation part 5a, and it transmits to terminal Tb, 
via circuit CCT. 

[0031]After terminal Tb, separates and takes out image data from the data sent from image 
encoding apparatus ICa, by the multiplexing separation part 5b, it decodes the image data by the 
decoding processing section 9b, and outputs the decoded image 10b, to the video output part 
1 2b. If the operator has set up a child screen display at this time, the image data 3b inputted 
with video input section 2b will be reduced to the set-up size, it will superimpose on the position 
to which the decoded image data 10b was set as the child screen 14b, and output displaying will 
become possible. 

[0032]When an operator sets up change of child screen-display 16b, using the **** input part 
17b, The display position and size information of a child screen which were newly set up 
promptly are inputted into the multiplexing separation part 5b, are multiplexed with the data of 
others, such as image coded data, and are transmitted to image encoding apparatus ICa,, and 
the display of the child screen newly set up by the same processing is attained. 
[0033]Next, when coding the image data 3a, the details of coding processing section 6a, the 
code amount of the child screen area 8a, is made to decrease as compared with field 7a, of the 
other same area are explained, referring to drawing 2 . Identical codes are given to the coding 
processing section and identical configuration which were shown by drawing 7 , and explanation is 
omitted. 

[0034] Drawing 2 is a block diagram of coding processing section 6a, in this embodiment, and the 
code amount of the child display part of an image decoder is controlled by controlling quantizing 
part 604 , by control-section 4a,. Namely, if the conversion factor which carried out orthogonal 
transformation of the block which contains a child display part from the orthogonal 
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transformation section 603 is inputted into quantizing part 604 ,, Control-section 4a 1 quantizes 
by quantization step width, such as twice of the usual quantization step width, bigger than the 
usually used quantization step width. 

[0035]The quantization step width of the block containing the above-mentioned child display 
part, It may be made for control-section 4a, to control to use the quantization step width (for 
example, quantization step width is 5 addition etc. to the twice of 20 and the usual quantization 
step width, or the usual quantization step width) which the operator inputted from operation 
input section 17a,. 

[0036]As mentioned above, by quantizing the block containing a child display part by bigger 
quantization step width than usual, many of conversion factors are set to 0, and the code 
amount at the time of being coded by the variable length coding section 610 decreases. Since 
the setting variation of an operator (a user, a serviceman) becomes possible about quantization 
step width at arbitrary stages (beforehand inside of **********), suitably, it unites with taste, 
change becomes possible, and adjustment of image quality, etc. can be performed with more 
user-friendly pictorial communication equipment, for example, 

r0037l Drawing 3 (a) is a flow chart in reception of the child screen position and large **** 
information by the side of image encoding apparatus ICa,. If the information etc. which detect 
reception of child screen information (Step (it is hereafter written as "S") 1), and make small the 
child screen position and size from image decoding device IDb, are always received in the image 
encoding apparatus ICa, side, The information is written in the memory for memory in control- 
section 4a, which is not illustrated (to overwrite or another field) (S2), and it is based on the 
new child screen information, and the coding data of a child display part is controlled and it 
transmits (S3, S4). And based on the newest child screen information written in the memory for 
memory in the control-section 4a, which is not illustrated, the coding data of a child display part 
is controlled until it receives the information which changes the following child screen position, 
size, etc. (S1,S3, S4). 

[0038] Drawing 3 (b) is a flow chart of transmission of the child screen position and size 
information by the side of image decoding device IDb,, After transmitting an initial value to image 
encoding apparatus ICa, (S10), the setting variation by an operator is always detected (S1 1), If a 
setting variation is not performed, when nothing is done, but it carries out loop processing of 
Step S11 and an operator performs new setting out, the information is transmitted to image 
encoding apparatus ICa, (S10). When not displaying a child screen, the information that there is 
no size of a child screen is transmitted. 

[0039]As explained above, according to image encoding apparatus ICa, of a 1st embodiment By 
controlling the code amount of field 8a, equivalent to a child screen to become less than other 
field 7a,, it becomes possible by decreasing the code amount per screen to shorten a 
transmission time. 

[0040]Since a child display part is not removed also when a child screen display position and a 
size change, an image can be prevented from completely being lost in the position which had a 
child screen before in a decoded image. 

[0041]When the display position and large **** information on a new child screen are received 
while image encoding apparatus ICa, had transmitted image data, Realizing reduction of the code 
amount per screen continuously by controlling the code amount of the field of the inputted 
image equivalent to a new child display part to become less than other fields, the image quality 
of the portion which had the child screen before improves. As mentioned above, when coding the 
image data 3a, the code amount of the child screen area as a means it is made to decrease as 
compared with the field of the other same area, Although control-section 4a, quantized in a 1st 
embodiment by bigger quantization step width (for example, twice of the usual quantization step) 
than the usually used quantization step width, other means are explained below. 



JP,2001-045476,A [DETAILED DESCRIPTION] 



6/7 



[0042]It explains focusing on a different part with said 1 st embodiment, referring to drawing 4 f or 
a 2nd embodiment of this invention. Identical codes are given to the above mentioned 
composition and identical parts, and explanation is omitted. DrawingjIJs a block diagram showing 
the relation between the video input section 2a, coding processing section 6a 2 , and control- 
section 4a 2 , The switch SW is formed between the video input section 2a and the subtraction 
part 603 of coding processing section 6a 2 , and turning on and off of this switch SW is considered 
as the composition which control-section 4a 2 controls. 

[0043]Namely, as a means which lessens the code amount of a child screen area in this 
embodiment as compared with the field of the other same area, Control-section 4a 2 controls 
coding frequency for between the video input section 2a-subtraction part 603 by the change of 
the switch SW, and controls the code amount of the child display part 61) of coding processing 
section 6a 2 . 

[0044]If the block inputted from the video input section 2a is a block containing the child display 
part 611, control-section 4a 2 will repeat turning on and off for the switch SW, and will make 
lower than other blocks the frequency where the data of the block is processed. It realizes by 
this performing coding processing for every regular frame, and not carrying out with other 
frames. What is necessary is just to code generally, in the range in which a coding interval is 
smaller than this frame number, since it has decided that a block must be coded once to a 
certain frame number in advice of the coding mode. 

[0045]By the above processing, if it integrates with the code amount of each block on a time- 
axis, respectively, the code amount of the code amount of the block containing the child display 
part 611 will become less than block 608 2 which does not contain a child display part. Although 
processing of orthogonal transformation or motion estimate generally has a large throughput, it 
becomes possible to exclude those processings and it does the effect of throughput mitigation 

[0046]The coding frequency at the time of the block with which the block inputted from the 
video input section 2a contains the child display part 61 1 , It may be made for control-section 
4a 2 to control to use the frequency (it is 1 etc. time etc. except a child display part to the 
frequency of 1 /2, or 3 seconds) which the operator inputted from operation input section 1 7a, 
(refer to drawing 1) , Thereby, since the setting variation of an operator (a user, a serviceman) 
becomes possible about coding frequency at arbitrary stages (beforehand inside of **********), 
suitably, it unites with taste, change becomes possible, and adjustment of image quality, etc. can 
be performed with more user-friendly pictorial communication equipment, for example. 
[0047]Next, it explains focusing on a different part with said 1st and 2nd embodiment, referring 
to drawing 5 for a 3rd embodiment of this invention. Identical codes are given to the above 
mentioned composition and identical parts, and explanation is omitted. Drawing 5 is a block 
diagram showing the relation between the video input section 2a, coding processing section 6a 3 , 
and control-section 4a 3 , The filter 613 is formed between the video input section 2a and the 
subtraction part 602 of coding processing section 6a 3 , and this filter 613 is considered as the 
composition which control-section 4a 3 controls. That is, control-section 4a 3 controls the code 
amount of the child display part 611 of image decoding device IDb 1 ( drawing 1 (b)) by controlling 
the filter part 61 3 for filtering the inputted image from the image input part 2a. 
[0048]If the block which inputs control-section 4a 3 from the video input section 2a by the 
above-mentioned composition is a block containing the child display part 611, the data of the 
block will be filtered by the filter part 613. The filter part 613 used here can consider the 
smoothing filter of a picture. In order to generally change the picture element data of a block into 
a frequency axis in the latter orthogonal transformation section 603, The ingredient of block data 
smoothed by the filter part 613 with high frequency decreases, Many of conversion factors after 
the high frequency component of the conversion factor outputted from the orthogonal 
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transformation section 603 as a result became a small value and was quantized by the quantizing 
part 604 are set to "0", and the code amount at the time of being coded by the variable length 
coding section 610 will decrease. 

[0049]The filter at the time of the block with which the block inputted from the video input 
section 2a contains the child display part 61 1, It may be made for control-section 4a 3 to control 
to use the filter specifications (the smoothing filter of strong intensity, or the smoothing filter of 
weak intensity) which the operator inputted from operation input section 17a, ( drawing 1 (a)). 
Thereby, since the setting variation of an operator (a user, a serviceman) becomes possible 
about filter specification at arbitrary stages (beforehand inside of **********), suitably, it unites 
with taste, change becomes possible, and adjustment of image quality, etc. can be performed 
with more user-friendly pictorial communication equipment, for example. 
[0050] 

[Effect of the Invention]Even if it performs change and elimination of a child screen of a position 
and a size after ** by the image encoding apparatus of this invention, and an image data 
transmission start as explained above, the portion which does not have an image into a decoded 
image is not made. Therefore, offer of the position of a child screen display and an image 
encoding apparatus with little image quality deterioration by change of a size was attained, the 
code amount of the following image data not becoming large, or the state where image quality is 
bad not continuing for a long time, and decreasing the code amount per screen. The code 
amount reduction control by a control section is enabling setting out or change of the code 
amount of said child screen display region data, or the decrement of a code amount arbitrarily. 
Since the setting variation of an operator became possible at arbitrary stages, adjustment of 
image quality, etc. were suitably united with taste, for example, it became possible, and offer of a 
more user-friendly image encoding apparatus was attained. 



[Translation done.] 



